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Against the Odds: The Impact of the Key Communities at Colorado State 
University on Retention and Graduation for Historically 
Underrepresented Students 


Abstract 

Learning communities are a high impact activity that can influence students’ likelihood for success. Colorado 
State University (CSU) created the Key Communities (Key) program, which is open to all students but 
targets students that have persistently lower graduation and retention rates. The majority of Key students are 
under-represented (ethnically diverse, low-income, and/or first generation to college) and/or students with 
lower levels of academic preparation. This paper describes the structure and purpose of Key and shares the 
results of an institutional level assessment of Key’s impact on graduation and retention. Since participation in 
Key is not randomly assigned, this analysis utilizes propensity score matching to estimate Key’s treatment 
effect. Results show that Key has a positive impact on graduation and retention for all students, but Key is 
incredibly effective for students who come to CSU with characteristics that have historically put them at risk 
for attrition. 
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Introduction 

Among all students enrolled at public 4-year institutions, about 80% are 
retained to the second fall semester and only 59% graduate within 6 years (Kena, 
et al., 2014). Students from underrepresented backgrounds (first generation, low 
income, and ethnically diverse) have even lower retention and graduation rates 
(Engle & Tinto, 2008). Furthermore, the retention and graduation gap for students 
from underrepresented backgrounds has increased significantly over the last 20 
years (Kena, et al., 2014). In order to address the widening inequality in 
undergraduate success, interventions that are targeted for students who have been 
historically underserved by higher education must be considered. 

As a high impact activity that can increase a student’s likelihood to succeed 
(Tinto & Goodsell, 1994), learning community participation has a stronger 
positive impact for historically underrepresented students (Kuh, Kinzie, Buckley, 
Bridges, & Hayak, 2006; Zhao & Kuh, 2004) because students’ critical first-year 
needs are addressed (Ward, Siegel, & Davenport, 2012; Lardner, 2004). Learning 
communities connect students to each other, to campus resources, and to 
intentionally integrated learning experiences that make a significant difference in 
students’ persistence, learning, and views of themselves (Brownell & Swaner, 
2010). However, due to a lack of culturally appealing activities (Hawkins & 
Larabee, 2009), underrepresented students typically do not choose to become 
involved in learning communities. Therefore, in order to be effective, programs 
must be designed specifically for these students. 

Colorado State University (CSU) implemented the Key Communities (Key) 
in 1998 as a way to restructure the first-year experience, particularly for students 
from historically underrepresented populations. This paper describes the purpose, 
principles, and structure of Key, reviews an institutional level assessment of 
Key’s impact on graduation and retention, and concludes with implications for 
learning community practice. 

History and Purpose of the Key Communities 

In the mid-1990s, CSU analyzed student retention and persistence data 
focused on the outcomes for underrepresented students. Similar to trends 
documented in postsecondary scholarship, this analysis revealed that students 
from underrepresented backgrounds are retained and graduate at lower rates than 
their peer groups. In response, we designed Key as an intervention strategy 
focused on the critical first year for all students and students most at-risk for 
attrition. Following established retention theory, Key is built on the premise that 
structured first-year programs are effective in helping underrepresented students 
succeed (Thayer, 2000). 
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Key is grounded in Tinto’s (1994) theories on student retention, practices 
for a structured first year experience by Muraskin (1998) as well as concepts and 
lessons learned from the TRIO program. Established by the Higher Education Act 
of 1965, TRIO is successful in meeting the needs of first-generation and low- 
income students within the educational environment by providing a 
comprehensive, structured, specialized experience that supports educational 
access and retention (http://www2.ed.gov/about/offices/list/ope/trio/index.html). 

Key creates a community with frequent and rewarding contact among 
faculty, staff, and students in a variety of settings, both inside and outside the 
classroom. Our approach follows the three domains of retention essential to first- 
generation student success: academic integration and the importance of 
reconciling the gaps between students’ expectations and realities; personal and 
social integration to foster a sense of belonging; and cultural integration so 
students can quickly understand the values, norms, traditions, and beliefs of the 
campus culture (Ward, et al., 2012). Key’s goals include increasing academic 
performance, increasing retention and graduation rates, fostering active 
engagement and campus involvement, increasing diversity awareness and 
understanding, and creating a sense of community and satisfaction among 
participants. Fundamentally, Key aims to increase retention and graduation rates 
beyond what would be predicted for students based on entering demographic and 
academic characteristics. 

The Key Communities: Guiding Principles and Program Structure 
Guiding Principles 

Tinto (1994) asserts that programs must be committed to the students they 
serve, stating “the essential character of such communities lies not in their formal 
structures, but in the underlying values which inspire their construction” (p. 146). 
The overall structure of Key is inspired by four principles that are grounded in 
theory and serve as a prompt to guide decision-making and program growth. 

Design with Diversity in Mind 

Key is intentional about creating a positive educational experience with 
attention to underrepresented students, who comprise the majority of the Key 
student population. Strategies designed specifically for first generation and low- 
income students are likely to be successful for the general population, whereas 
strategies designed for the general population—without attention to 
underrepresented students—will often miss meeting their unique needs (Thayer, 
2000). Because students are entering an environment that is drastically different 
from their own homes (Rendon, Garcia, & Person, 2004), designing with diversity 
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in mind addresses negative experiences such as low expectations, inadequate high 
schools, and perceptions that students are deficient while simultaneously offering 
an experience that is academically rigorous, compassionate, nurturing, 
empowering, liberating, and democratic (Rendon, et al., 2004). 

Maximize Learning Opportunities 

Key connects students to in- and out-of-class activities in a manner that does 
not isolate students' learning experience to any one realm. Organizing student 
learning across the curriculum and co-curriculum is a core tenet of learning 
communities, while building-in support for students academically and socially is a 
condition that promotes student success (Tinto, 2012). The community is 
grounded in the curriculum with intentionally designed co-curricular experiences 
that are essential for student learning (Smith, MacGregor, Matthews, & 
Gabelnick, 2004). Furthermore, because faculty design the integrated curriculum, 
deeper understanding of the course material emerges along with deeper 
connections with students and staff in the community (Shapiro & Levine, 1999). 

Voice a Powerful Message 

Muraskin (1998) discusses the importance of voicing a powerful message of 
success when structuring the first year of college for underrepresented students. 
Providing clear and consistent expectations on the requirements to succeed and 
holding high expectations for students’ abilities to meet those requirements are 
additional conditions that promote student success (Tinto, 2012). In Key, the 
message to students is that they will be successful by attending class, participating 
in what is offered, and meeting all expectations. 

Give Honest Feedback Early and Often 

Key recognizes that student behaviors signaling potential difficulty need to 
be identified at the earliest possible time. Effective intervention is based on honest 
and timely feedback that informs students of their academic and social standing. It 
is also important to acknowledge that first-generation college students have less 
information and context about the university environment and expectations, so 
information and feedback is critical to success. Since the single most important 
means of establishing a connection at the university is to be attached to one 
person (Levitz & Noel, 1989), Key puts students in contact with mentors and full¬ 
time staff who care about them as a whole person, connect with them 
individually, and help them make the transition to and through college. 
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Program Structure 

In addition to the four guiding principles, Key draws on research about 
successful program design. Levine and Shapiro (2000) state that effective learning 
communities share several basic characteristics: 

...they organize students and faculty into smaller groups; encourage 
integration of the curriculum; help students establish academic and 
social support networks; provide a setting for students to be socialized 
to the expectations of college; bring faculty together in more 
meaningful ways; focus faculty and students on learning outcomes; 
and provide a setting for community-based delivery of academic 
support programs (p. 14). 

Key is intentional about engaging students in the academic and social 
experiences of college, and the program is structured to maximize the level of 
engagement and integration that occurs. The 475 first-year students participate in 
one of five communities: Key Academic; Key Culture, Communication, and 
Sport; Key Explore; Key Health Professions; and Key Service. All five 
communities are grounded in the values of academics, leadership, diversity, 
service, and community. Each community ranges from 75-150 students, and each 
is further organized into smaller clusters of 19 students. The program also 
employs several additional critical strategies for promoting success. 

Key Recruitment and Orientation 

Since a high proportion of the population is first-generation to college, low- 
income, and/or ethnically diverse, Key implements a comprehensive and 
intentional recruitment process designed to involve students who are otherwise 
not likely to participate. Recruitment focuses on diverse student populations and 
emphasizes that the program is an honorary experience with high expectations. 

Fostering a sense of community, communicating high expectations, 
orienting students to the expectations of Key, and acquainting students with peers, 
faculty, and staff are all frontloaded during a two-day Key Orientation prior to the 
start of the fall semester. This is an important, early step: students hear the 
message that they belong at CSU, and they are also oriented to the expectations of 
Key and the university experience. 

A Shared Residential Experience 

Kuh (2005) states that institutions that are serious about student engagement 
must structure the first year so students spend time with peers, preferably by 
living on campus. The residential component of Key is consistently rated as one 
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of the most beneficial aspects of the students’ overall experience. All first-year 
Key students live together in a popular, renovated, and centrally located hall. This 
centrality puts students at the academic core of campus where they have access to 
faculty and staff offices, classrooms, the library, and the student center. 
Furthermore, a renovated hall provides the best financial option by keeping the 
prices affordable. Considerations like this minimize isolating students who may 
already be marginalized on campus. This design also illustrates how students’ 
needs are in the forefront for decision-making about all aspects of the Key 
program. 

Integrated Course Clusters and Key Seminar 

The Key cluster structure is similar to learning community models that involve a 
freshman seminar with one to two university core curriculum courses that are 
unmodified in design (Smith, MacGregor, Matthews, & Gabelnick, 2004). The 
practice of course clustering helps mitigate isolation and enables students to make 
friends in an otherwise often impersonal setting (Muraskin, 1998). The term 
“cluster” defines each smaller cohort of 19 students in a shared curricular 
experience organized around a central theme. For instance, the Psychobiology 
course cluster is represented in Figure 1. 


Psychobiology 



Figure 1 Psychobiology Cluster 

In this cluster, Key students have seats reserved in Attributes of Living 
Systems (LIFE 102) and General Psychology (PSYCH 100) and are the only 
students enrolled in the Key Seminar course (KEY 192). The seminar is designed 
to help students integrate behavioral and biological understandings of the human 
experience and use these understandings to enhance their ability to make choices 
about their own behavior. Key Seminars are intended to be intellectually 
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challenging academic experiences, to foster substantive interactions between 
individual students and faculty and between and among student peers, and to 
teach basic competencies through active learning experiences that promote critical 
thinking skills. 

Leadership and Staffing Model 

The three critical staffing positions include undergraduate Key mentors, 
full-time Key coordinators, and Key seminar faculty. Key mentors are students 
who are in good academic and judicial standing, have experience working with 
diverse student populations, know about campus resources, and have strong 
interpersonal communication and academic skills. The full-time Key 
coordinators’ primary responsibilities include supervising mentors, assisting with 
the development and implementation of the program, connecting students to 
campus resources and opportunities, and ensuring that the program activities are 
timely, high quality, and consistent with Key’s intended goals and philosophies. 
In addition to developing and teaching the seminar, Key seminar faculty involve 
and supervise their assigned Key mentor as an undergraduate teaching assistant, 
conduct an orientation session with students during Key Orientation, and provide 
feedback on student performance. 

Early Warning System 

One method of providing early, often, and honest feedback to students is 
through an early warning and intervention system that gives students grade 
feedback and comments on their performance. Helping students stay on track by 
monitoring student progress enables them to take action before it is too late 
(Engle & Tinto, 2008). Key mentors use this feedback to facilitate a mid-semester 
conference with each student to discuss performance in classes, overall transition 
to the university, and creation of an action plan for learning effectiveness. 

Traditions for Community Building 

Finally, being intentional about fostering community in learning 
communities is important. Key does this in a variety of ways, including signature 
events where all 475 students come together in a community building activity. An 
example is Key Community Challenges, a day of field games, trivia, and sports 
during the fall semester, during which each cluster forms a team, designs T-shirts, 
and spends a few hours engaging in friendly competition. Finally, at the end of the 
academic year. Key sponsors a closing barbeque and awards celebration. 
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Research Purpose 

Key’s guiding principles and program structure are designed to maximize 
student learning and increase graduation and retention rates of students, 
specifically those from underrepresented backgrounds in higher education. The 
following section reviews an institutional level analysis that Key used to assess 
how well the program is meeting its goals. The purpose of this assessment was to 
provide evidence of Key’s influence on retention to the second-fall (second-year 
retention) as well as longer term, bearing on eventual graduation. A common 
critique of first-year learning communities, like Key, is that they may impact 
retention to the second year, but that the positive experiences do not last through 
graduation. However, this study provides compelling evidence that Key does have 
an immediate impact on second-year retention and a lasting impact on eventual 
graduation. 


Research Questions 

This institutional-level analysis compares the second-year retention and 
graduation rates among first-time students who participate in Key compared to 
those who do not. Additionally, this study focuses on understanding the 
differential impact of Key for historically underrepresented students by 
identifying a statistically and substantively significant interaction between a 
student’s likelihood to be in Key and the treatment effect of Key on graduation 
and retention. Therefore, two research questions guided this work. 

1. Does participation in the Key Communities have a positive impact on a 
student’s likelihood of being retained to the second year and graduating? 

2. Does the impact of participating in Key vary based on a student’s 
demographics and academic preparation? 

Data 


First-time, full-time students from the fall 2005 (FA05) through fall 2011 
(FA11) cohorts are included in this study. All of these cohorts are included in the 
second-year retention analyses; however, the cohorts included in the graduation 
outcomes are limited by the time frame of having the opportunity to graduate in 
four, five, or six years. Students attain Key status by participating in Key during 
their first academic year. Over the seven cohorts included in this study, Key has 
grown 85%, and the total first-year student cohort size has grown 16%. On 
average, Key served about 7% of the total first-year student cohort. Appendix A 
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provides the cohort counts for Key and non-Key students during this study’s time 
period. 

Methodology 

This study uses a combination of descriptive analyses to describe the 
demographics and academic preparation of Key and non-Key students as well as 
the observed graduation and retention rates. When proportions are compared, a 
chi-squared test is used to assess statistically significant differences and when 
means are compared a t-test is used. 

To address the first research question, a propensity score analysis is used to 
estimate the treatment effect of Key on second-year retention and graduation. All 
analyses are completed using Stata version 10. This type of quasi-experimental 
analysis is necessary because students self-select (are not randomly assigned) to 
participate in Key, which introduces statistical bias to causal inteipretations of 
observational data (Murnane & Willet, 2011). Propensity score matching allows 
for a comparison of demographically similar groups of students (Key/Non-Key) 
with the goal of understanding what the outcome would be for a Key student if 
he/she had not participated in the program. To estimate this counterfactual 
outcome, the propensity score approach uses a logistic regression model to 
calculate the probability that a student will be in Key based on a variety of 
academic and demographic variables. The probability of a student being in Key is 
his/her propensity score. The propensity score is then used to balance the dataset. 
In other words, every Key student is matched (based on having a similar 
propensity score) to a statistically similar non-Key student. The premise of this 
approach is that the matched non-Key students are a comparable control group to 
the Key students; therefore, the graduation or retention rates for these matched 
non-Key students are used to estimate the rates that would have been expected for 
a Key student if he/she had not been in Key (the counterfactual) (Guo & Fraser, 
2010 ). 

To address the second research question, a logistic regression is run on the 
balanced/propensity score adjusted data file with the propensity score, Key 
participation, and product of participation by propensity (interaction term) 
included as covariates and six-year graduation or retention as the outcome. The 
logistic regression model is then used to obtain the predicted probabilities of 
second-fall retention so that the difference in predicted probability by Key 
participation at every propensity level can be displayed graphically. 
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Variables 

To obtain a propensity score for the likelihood of being in Key, a variety of 
demographic and academic variables are used in a logistic regression model that 
predicts Key participation. In terms of academic variables, the student’s CCHE 
index (index) and college major are used. Index score, a measure of high school 
academic preparation, is a continuous variable specific to Colorado. This 
composite score is derived from high school GPA or high school rank and ACT or 
SAT test scores. More information regarding index can be found on Colorado’s 
Department of Higher Education website (http://highered.colorado.gov) . 

CSU has eight academic colleges that are entered in the model as dummy 
variables with undeclared students as the reference category. The propensity score 
model also includes five demographic variables. Minority status is a binary 
variable, with minority students compared to non-minority (white, unknown 
ethnicity, and international) students as the reference category. First generation 
and Pell recipients are both compared to their respective reference group. First 
generation status is self-reported, based on the student’s response to an 
admissions application question that asks if they are the first in their family to 
attend college. Pell recipient status is based on financial aid records for whether or 
not the student received a Pell grant his or her initial year. Residency represents 
whether or not the student was a Colorado resident (in-state) for tuition purposes 
during the initial year, and resident students are compared to the reference group 
of nonresidents. Gender is also included as a demographic covariate. Females are 
compared to the reference group of males. From prior institutional level analyses, 
all of these covariates are associated with graduation and retention. 

Results 

The results from this institutional-level assessment of Keys’ impact on 
graduation and retention are presented in the following two sections. First, 
descriptive statistics of Key and non-Key students (prior to any propensity score 
adjustments) are discussed. The results of the propensity score analysis follow. 

Descriptive Results 

Demographically and academically, Key students are different compared to 
non-Key students. Appendix B presents the proportions of Key and Non-Key 
students across the academic and demographic variables included in the 
propensity score models. 

As shown in Appendix B, the Key group includes statistically significant 
larger proportions of first generation, Pell recipient, minority, and female students 
compared to the non-Key group. There is not a statistically significant difference 
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in the proportion of Colorado residents across Key status. Appendix B also shows 
the lower average index score for Key students compared to non-Key students; 
the magnitude of this difference is very large. Additionally, Key students are more 
likely to enter CSU as an undeclared major compared to non-Key students. 

Table 1 displays the observed second-year retention and graduation rates for 
Key and Non-Key students averaged over multiple cohorts prior to the propensity 
score adjustments. 


Table 1 

_ Unadjusted Graduation and Retention Rate Comparisons, Key Participants vs. Non-Key 



Second-Fall Retention 1 

4-Year Graduation 2 

5-Year Graduation 3 

6-Year Graduation 4 

Key 

88.7% 

38.5% 

60.8% 

66.2% 

Non-Key 

83.2% 

37.7% 

59.9% 

64.1% 

Difference 

5.53% 

0.83% 

0.84% 

2.06% 

N for Key / Non-Key 

1991 / 27113 

1000/15225 

701 /11254 

408 / 7295 


1 Includes students from FA05 through FA11 first-time, full-time cohorts 

2 Includes students from FA05 through FA08 first-time, full-time cohorts 

3 Includes students from FA05 through FA07 first-time, full-time cohorts 

4 Includes students from FA05 and FA06 first-time, full-time cohorts 


As shown in table 1, Key students are retained to the second year at a rate 
that is 5.5 percentage points higher than non-Key students (X“=44.3, pc.OOO). 
However, Key graduation rates (4-, 5-, and 6-year) are not statistically different 
compared to the graduation rates of non-Key students. 

Propensity Score Adjusted Results 

As discussed in the methodology section of this report, propensity scores are 
used to create an appropriate comparison group of non-Key students. Appendix C 
contains the logistic regression coefficients for the models that are used to obtain 
the propensity scores. These models show that minority students, Pell recipients, 
and females have a positive association with Key participation. Index score is 
negatively associated with Key participation. 

Research Question #1: Average Treatment Effect 

Using the propensity score adjusted data set, the average treatment effect of 
Key is calculated by subtracting the graduation or retention rates for non-Key 
from the Key rates. Table 2 provides the graduation and retention rates for Key 
students and matched non-Key students. 
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Table 2 


Propensity Score Adjusted Graduation and Retention Rate Comparisons, Key Participants vs. Non-Key 



Second-Fall Retention 

4-Year Graduation 

5-Year Graduation 

6-Year Graduation 

Key 

88.7% 

38.5% 

60.8% 

66.2% 

Non-Key 

80.4% 

37.1% 

55.6% 

56.4% 

Difference (se) 1 ' 2 

8.29% (1.14%)* 

1.40% (2.17%) 

5.14% (2.63%)* 

9.80% (3.40%)* 

N for Key/Non-Key 

1991 /1991 

1000/1000 

408 / 408 

408 / 408 

Additional Students Retained or 
Graduated 

165 

NA 

NA 

40 


Average treatment effect among the treated, with standard error in parentheses 
2 *p<0.05 

Among the balanced data set that uses propensity scores to match non-Key 
student to Key students. Table 2 shows there is a gain of 8.3 percentage points in 
freshmen retention and increases in 4-, 5-, and 6-year graduation rates estimated 
at 1.4 percentage points, 5.1 percentage points, and 9.8 percentage points 
respectively. The second-fall retention, 5-year graduation, and 6-year graduation 
effect estimates are statistically significant. 

A benefit of the propensity score research design is that it allows for the 
application of the average treatment effect to estimate the actual numbers of 
additional students retained or graduated because of the treatment (Key). Using an 
average treatment effect to causally infer additional students retained or graduated 
is a not uncommon application of propensity score analyses for program 
assessment at the institutional level (Keller & Lacy, 2013). With respect to this 
study, the 8.3 percentage point increase in second year retention resulted in 165 
(.0829*1991) additional Key students from the FA05 to FA11 cohorts returning 
for their second year. Similarly, 40 (.098*408) additional students from the FA05 
and FA06 cohorts graduated in 6 years because of their Key participation. 

Additionally, the 9.8 percentage point increase in 6-year graduation rates 
can be used to estimate the impact Key has on CSU’s overall graduation rate for 
the FA05 and FA06 cohorts. The FA05 overall cohort includes 3,807 students, 
with a Key cohort of 190; therefore, an additional 19 students (190*.098) 
graduated within 6 years from CSU because of their participation in the Key 
program. The 19 additional graduates increased CSU’s FA05 overall rate by half 
of a percentage point (19/3,807)., The FA06 overall cohort is 3,971 students, with 
a Key cohort of 221; following the above logic, an additional 21 students 
graduated within 6 years from CSU because of their participation in the Key 
program. Key also increased CSU’s FA06 overall rate by half of a percentage 
point. 

Since each student in the adjusted dataset has a propensity (probability of 
the likelihood) for being in Key, we can describe the group of students who are 
likely to be in Key and the group of students who are unlikely to be in Key. For 
instance, students with a low likelihood of being in Key have a higher than 
average index (116) and aren’t very likely to be first generation, minority, or Pell 
recipients. Thus, the students with the highest propensity to be in Key are students 
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whose prior academic preparation and demographics are positively associated 
with attrition. Appendix D provides the descriptive statistics among students with 
the lowest (bottom quartile) and highest (top quartile) likelihood of being in Key. 


Research Question #2: Differential Treatment Effect 

The original purpose of Key is to decrease the graduation rate gaps for 
underrepresented populations. Therefore, it is also important to assess whether the 
treatment effect of Key varies based on a student’s likelihood to be a Key 
participant. Figure 2 graphs the probability of being retained for Key and matched 
non-Key students by their likelihood to be in Key. 


Figure 2 


100 % 

c 90% 

1 80% 
a> 

1 70% 
? 60% 
g 50% 
! 40% 

O 

130% 
| 20 % 
“■ 10 % 


Impact of Key Participation on Second-Fall Retention by 
Probability of Key Participation 


o% 

o% 


5 percentage point gap at lowest probability 
of beirg in Key and a 16 percentage point gap 
at the highest probability of being in Key 


■Key 


Non-Key 


4% 


5% 


6% 8% 15% 19% 

Probability of Being in Key 


22% 26% 33% 


The impact of the Key program is differentially higher for traditionally 
underrepresented students (those with the highest probability of being in Key). In 
figure 2, the x-axis shows the range of probabilities for the likelihood of being in 
Key, while the y-axis displays the probability of second-fall retention. Also 
shown in table 6, students with a low probability of being in Key are students with 
historically higher rates of success. For these students, the treatment effect of Key 
is relatively small. Participation in Key increases their probability of being 
retained by five percentage points. However, as a student's probability of 
participation in Key increases, so does the treatment effect. For a student with a 
high likelihood of being in Key, there is a 16 percentage point increase in the 
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predicted probability of being retained to the second year. Key has a positive 
effect on graduation for all students; however, Key has a much larger effect for 
traditionally underrepresented students—those who are more likely/have the 
highest propensity to be in Key. 

The effect of Key participation on graduation is also dependent on the 
likelihood of Key participation. Similar to the results shown in Figure 2 for 
retention, the gap in the predicted probability of graduating within 6 years 
between Key and non-Key students is smallest for students less likely to be in 
Key and largest for students most likely to be in Key. 

Interpretations 

Key students are more likely than the general population to be a minority, 
first generation, or a Pell recipient. Additionally, Key students tend to have lower 
levels of academic preparation. At CSU the retention rates for Key students are 
statistically higher compared to non-Key students; however, the graduation rates 
of Key students are very similar to non-Key students. This bivariate approach 
cannot be used to evaluate the impact of Key on graduation or second-year 
retention because it does not account for demographic and academic differences 
between Key and non-Key students. Therefore, propensity scores are used to 
match non-Key students to Key students in order to assess a treatment effect of 
Key on graduation or second-year retention. This is done by comparing the 
second-year retention or graduation rates of Key students to the second-year 
retention or graduation rates of the matched (demographically similar) non-Key 
students. 

The more nuanced propensity score analysis is an important assessment tool 
in this study. Key serves students who typically have lower graduation rates. The 
observed graduation rates of Key students are similar to non-Key students, which 
(if demographics and academic preparation are ignored) could cause some to 
question the effectiveness of Key. The additional propensity score analysis allows 
us to identify a treatment effect of Key on participants, making a much stronger 
case for the program’s efficacy. Additionally, calculating the treatment effect 
allows us to estimate the number of additional students retained/graduated 
because of the program and thus the impact of Key on CSU’s overall retention 
and graduation rates. This type of evidence can be used to justify continued 
institutional support. 

In summary, participation in Key appears to support student success by 
mitigating the negative effects of lower academic preparation and at-risk 
attributes on graduation and retention. Key has a positive effect on graduation and 
retention for everyone, but Key participation differentially impacts the likelihood 
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of retention and graduation for first generation, minority, and Pell recipients or 
students with lower levels of academic preparation. 

Implications for Learning Communities Research and Practice 

This study provides compelling evidence that Key has both an immediate 
and long-term impact on CSU student retention and graduation rates. It also has 
broader implications for learning communities’ research and practice beyond our 
institution. 

Retention and Graduation Rates 

Learning communities are often implemented as a strategy to meet retention 
and graduation goals. Many programs have observed higher retention and 
graduation rates, yet attributing those positive outcomes directly to a learning 
community presents a challenge. For one, it is often hard to find an appropriate 
control group to compare the learning community treatment. Also, there is a 
question of self-selection bias, whether students who take advantage of the 
learning community are those who are most motivated and would have succeeded 
anyway. Given that learning community participation is posited to have greater 
impact on underrepresented students (Kuh, et al., 2006; Zhao & Khu, 2004), this 
study leaves practitioners and administrators with evidence of the positive impact 
that an intentionally-designed comprehensive learning community has on 
retention and graduation rates, particularly among students who have historically 
been the most at-risk for attrition. This outcome is important given institutions’ 
concern with increasing retention and graduation rates as well as reducing 
graduation rate gaps among historically underrepresented students. 

Considerations of Learning Community Design 

Considerations of design are essential when using learning communities as 
an intervention strategy for student success. In this case, Key has statistically 
significant larger proportions of first generation, Pell recipient, and ethnically- 
diverse students. Designing a learning community for this population ensures that 
the intervention meets these and all students’ needs. The design of Key responds 
to the research that suggests that a learning community that targets 
underrepresented students should scale down the overall college experience and 
provide personalized attention from dedicated staff (Engle & Tinto, 2008). Key is 
also effective because it harnesses equity-oriented pedagogical principles, is 
designed with diversity in mind, and does not marginalize students through being 
identified as a “minority serving” program per se. A future research trajectory is 
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to consider an in-depth qualitative and ethnographic analysis that captures the 
practices of Key and how all involved make meaning of the experience. 

Scaling Up to Become an Institutional Intervention 

Many programs in higher education begin as small-scale efforts reaching a 
few students. The success of such programs, as Key has realized, will beg the 
question of whether or not these smaller scale efforts can grow large enough to 
become institutional interventions. In this case, increasing Key by 85% since 
2005 was accomplished in a way that maintained the principles and intensive 
structure, resulting in increased retention and graduation that positively impacts 
the overall graduation rates at CSU. Shifting away from the principles, structure, 
size, and intentionality that undergird smaller communities may dilute the overall 
impact on students; therefore, scaling up learning community efforts should 
proceed cautiously in ways that maintain the principles and comprehensive 
program structure. 

Collaborations Between Institutional Research and Learning Communities 

This study demonstrates the importance of collaboration between learning 
community leadership and other offices on campus, specifically Institutional 
Research. This type of collaboration is essential to develop research studies that 
can have important practical impact at the institutional level. In this case, it took 
expertise from both the Director of Key Communities as well as Institutional 
Research to design the current study. The results presented in this article have 
been extended to very practical applications at the institutional level. For instance, 
this study has been used to analyze a return on investment for Key. By estimating 
the tuition revenue generated from retaining students and factoring in the cost of 
running the entire Key cohort through the initial academic year, we have been 
able to show that the program pays for itself. 

Because assessing learning community impact needs to remain central to the 
overall effort, collaborations with Institutional Research should be considered 
among learning community practitioners as a best practice. Such research is 
critical to the field of learning communities so that programs can continue to 
argue that the impact they have is not compensatory or ancillary but in line with 
institutional learning goals. 


Conclusion 

By most measures in higher education, the students who have the highest 
propensity to participate in Key are those who have historically been the most at- 
risk for attrition. They are the first in their families to embark on a college degree. 
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They are low-income. They are students of color at a predominantly white 
institution. These students often receive the least attention, are held to the lowest 
expectations, and are lost in systems that all too often support the majority group. 

This study found that a learning community focused on underrepresented 
students has a positive impact on students’ likelihood of being retained to the 
second year and graduating and that the impact varies based on students’ 
demographics and academic preparation. Because Key appears to support student 
success by mitigating the negative effects of lower academic preparation and at- 
risk attributes on graduation and retention, traditionally underserved students 
benefit the most from participating.. These results demonstrate added value to 
institutions that are concerned with not only the quality of the student learning 
experience but also outcomes related to retention and graduation, particularly 
among underrepresented students. 

The students who participate in Key have beaten the odds by participating in 
an intentionally designed learning community that resulted in higher retention and 
graduation rates among all participants, with the greatest impact on those who 
have historically been the most at-risk for attrition. The real story of such 
communities, perhaps, may not be in the numbers. The real story may lie in the 
reality that this community is serving as a cultural home for students who all too 
often are found at the margins, and not the center, of the university experience. 
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Appendix A 

Count of First-time, Full-time Students by Cohort Term 


FA05 FA06 FA07 FA08 FA09 FA10 FA11 


Total 

Non-Key 3,617 3,750 

3,994 

4,006 3,903 

4,028 

4,068 

27,366 

Key 190 221 

294 

302 300 

341 

351 

1,999 

Total 3,807 3,971 

4,288 

4,308 4,203 

4,369 

4,419 

29,365 

Appendix B 






Descriptive Statistics of Key and Non-Key Populations 




Key 


Non-Key 




First Generation (%)* 

38% 

25% 




Pell Recipient (%)* 

32% 

16% 




Minority (%)* 

45% 

13% 




CO Resident (%) 

81% 

79% 




Female (%)* 

64% 

55% 




Undeclared (%)* 

33% 

27% 




Index (average)* 

111 

114 




* p<.05 






Appendix C 






Logistic Regression Coefficients for Key Community Participation 




Retention 

4-Year Graduation 5-Year Graduation 

6-Year Graduation 


Cohorts 1 

Chorts 2 

Chorts 3 

Chorts 4 



b(se)‘ 

5 



Minority 

1.47 (0.05)* 

1.42(0.07)* 

1.40 (0.09)* 


1.49 (0.11)* 

First Generation 

0.20 (0.05)* 

0.06 (0.07) 

-0.02 (0.09) 


-0.01 (0.12) 

Pell Recipient 

0.43 (0.06)* 

0.32 (0.09)* 

0.25(0.10) 


0.29(0.13) 

CCHE Index 

-0.01 (0.00)* 

-0.02 (0.00)* 

-0.02 (0.00)* 


-0.02(0.01)* 

CO Resident 

-0.06 (0.06) 

-0.23 (0.08) 

-0.25 (0.10) 


-0.27 (0.13) 

Female 

0.27 (0.05)* 

0.44 (0.07)* 

0.55 (0.09)* 


0.68 (0.12)* 

College Major at Entry 






Ag. Sci. 

-0.42 (0.13) 

-0.16(0.17) 

-0.15(0.21) 


-0.39 (0.30) 

App. Human Sci. 

-0.25 (0.08) 

-0.19(0.12) 

-0.05 (0.13) 


-0.08 (0.18) 

Business 

-0.02 (0.10) 

0.32 (0.14) 

0.35 (0.17) 


0.23 (0.22) 

Engineering 

-2.81 (0.32)* 

-2.87 (0.58)* 

-2.32 (0.59)* 


-2.10(0.72) 

Lib. Arts 

-0.02 (0.07) 

0.17(0.10) 

0.24(0.12) 


0.22(0.16) 

Natural Sci. 

-0.06 (0.07) 

0.14(0.10) 

0.07 (0.13) 


-0.07 (0.17) 

Vet. Med. 

-0.05 (0.13) 

0.15(0.17) 

0.09 (0.20) 


-0.21 (0.27) 

Natural Resources 

0.20 (0.14) 

0.72 (0.18)* 

0.90 (0.20)* 


1.17(0.24)* 


1 Includes students from FA05 through FA11 first-time, full-time cohorts 

2 Includes students from FA05 through FA08 first-time, full-time cohorts 

3 Includes students from FA05 through FA07 first-time, full-time cohorts 

4 Includes students from FA05 and FA06 first-time, full-time cohorts 

5 Cells display the regression coefficent with its standard error and an asterisk to indicate that p <.05 
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Appendix D 


Descriptive Demographics of Students by their Likelihood of Participating in Key 



Average 

Index 

Percent First 
Generation 

Percent 

Minority 

Percent Pell 
Recipients 

Percent 

Female 

Percent 

Resident Three Most Likely Colleges 

Lowest likelihood of 
being in Key (bottom 
quarter) 

116 

11% 

1% 

1% 

46% 

Undeclared (24%); Natural 
Sciences (19%); Liberal 
76% Arts (17%) 

Highest likelihood of 
being in Key (top 
quarter) 

106 

67% 

100% 

73% 

75% 

Undeclared (44%); Natural 
Sciences (22%); Liberal 
87% Arts (17%) 
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